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Image processing apparatus and method, and computer readable storage medium 



(57) Input image data is divided into blocks so as to 
make adjacent blocks partially overlap image data, and 
the divided image data is filtered in the unit of block. An 
image processing apparatus and method are provided 
which can process image data of an arbitrary size at 

fig: 1 



high speed independently from the capacity of a buffer 
memory, and a computer readable storage medium Is 
provided which stores a program realizing such a 
method. 
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Description 

BACKGROUND OF THE INVENTION 
Field of the Invention 

[0001] The present invention relates to an image 
processing apparatus and method suitable for perform- 
ing a filtering process and coding process of image 
data, and to a computer readable storage medium stor- 
ing process for realizing such a method. 

Relata Background Art 

[0002] Digital still cameras are widely used as an 
image pickup device for computers. An image compres- 
sion method utilizing DCT (discrete cosine transform) 
such as JPEG is used for digital still cameras. While dig- 
ital stilf cameras are prevailing, high speed continuous 
photographing and power saving have been desired. In 
order to shorten the time taken to pickup images and. 
record image data, the processes from signal process- 
ing to image corrpression have been conducted hereto- 
fore by hardware. 

[0003] With a conventional hardware configuration, a 
capacity proportional to the input image scan size in a 
horizontal direction is required for a buffer which is used 
for filtering accumulated charge information read from 
image pickup elements such as CCD in horizontal and 
vertical directions and for a buffer which is used for 
raster/block conversion of raster scan sequential image 
data into block scan sequential image data. For exam- 
ple, assuming that input image data has 1034 pixels in 
the horizontal direction and 770 pixels in the vertical 
direction, a horizontal filter has 1 1 taps, a vertical filter 
has 3 taps, a YUV sampling ratio of JPEG is 4:2:2, CCD 
image data has 1 0 bits, and each of YUV has 8 bits, the 
following data is determined: 

Hd = 1034, where Hd is the number of pixels of 
input image data in the horizontal direction; 
Hr = 1034 - INT(11/2) x 2 = 1024, where Hr is the 
number of pixels of image data in the horizontal 
direction output after signal processing (a filtering 
process, a YC separation, an edge process, a 
gamma process, and the like); 
a capacity of a buffer for filtering in the horizontal 
and vertical directions 
= 2 X Nd X 10 = 20680 (bits); and 
a capacity of a buffer for raster/block conversion = 8 
X Hr ((the number of bits of Y) + (the number of bits 
of UV)) 

8 X 1024 X 16 =s 131072 (bits). 

[0004] In the system using a plurality of image sizes, 
the buffer capacity has been determined conventionally 
in accordance with a maximum horizontal image size, 
respectively for a buffer for filtering in the horizontal and 



vertical directions and for a buffer for raster/block con- 
version. This Is not economical. 

[0005] Conventional techniques are associated with a 
problem that when a CCD sensor of non-square is 
5 used, the compressed image data has distortion. 

[0006] There is also a problem that since image data 
after signal processing is JPEG compressed, it is diffi- 
cult to resize the image data. 

10 SUMMARY OF THE INVENTION 

[0007] Under the above-described background of the 
invention, it is a concern of the present invention to pro- 
vide an image processing apparatus and method cape- 
rs ble of processing image data having an arbitrary size at 
high speed independently from the capacity of a buffer 
memory, and a computer readable storage medium 
storing pi-ocesses realizing such a method. 
[0008] According to one aspect of the present inven- 
20 tion. there is provided an image processing appara- 
tus/method wherein input image data is divided into 
blocks adjacent blocks of which partially overlap the 
image data, and the divided image data is subject to a 
filtering process in a block unit basis. 
25 [0009] According to an another aspect of the present 
invention, there is provided readable storage medium 
which stores a program executing steps of inputting 
image data and dividing the input image data into blocks 
adjacent blocks of which partially overlap the image 
30 data. 

[001 0] Othere advantages, feiatures and advantages 
of the invention will become apparent from the following 
detailed description taken in conjunction with the 
acconipanying drawings. 

35 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0011] 

40 Fig. 1 is a block diagram showing the structure of an 
image processing apparatus according to a first 
embodiment of the invention. 
Fig. 2 is a diagram showing image data stored in 
RAfvl 4. 

45 Rg. 3 is a diagram illustrating the operation of a 
two-dimensional DMAC. 

Fig. 4 is a diagram illustrating a method of dividing 
input data, the method being executed by a signal 
processing circuit. 
50 Fig. 5 is a diagram showing an effective image area 
of each divided area. 

Fig. 6 is a diagram illustrating an overlap of blocks 
in the horizontal direction. 

Fig. 7 is a diagram illustrating an overlap of blocks 

55 in the vertical direction. 

Fig. 8 is a diagram illustrating output data of the sig- 
nal processing circuit. 

Fig. 9 is a block diagram illustrating a clock control 
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of the signal processing circuit, the clock control 
being executed by a process SSG circuit. 
Fig. 10 is a block diagram showing the structure on 
an image processing apparatus according to a sec- 
ond embodiment of the invention. 
Fig. 1 1 is a diagram illustrating a phase relation of 
input and output data during enlargement and 
reduction. 

Fig. 12 is a block diagram showing the structure of 

an image processing apparatus according to a third 

embodiment of the Invention. 

Fig. 1 3 is a block diagram showing the structures of 

an enlarging and reducing circuit ar>d phase buffers 

according to the third embodimeint. 

Fig. 14 is a block diagram showing the fundamental 

structures of the enlarging and reducing circuit and 

phase buffers. 

Fig. 15 is a timing chart illustrating the operation of 
control signals output from a process SSG circuit. 
Fig. 16 is a block diagram showing the structure of 
an image processing apparatus according to a 
fourth embodiment of the invention. 
Fig. 1 7 is a block diagram showing the structures of 
an enlarging and reducing circuit and phase buffers 
according to the fourth embodiment. 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

[0012] Fig. 1 is a block diagram showing an image 
processing apparatus according to the first embodiment 
of the present invention. A ROM 14 is a storage medium 
for storing a program which is used by a CPU 5 to exe- 
cute a process to be described later. This storage 
medium may be a semiconductor memory, an optical 
disk, a magneto-optic disk, a magnetic medium or the 
like. These storage media may also be a non-volatile 
memory card, a DC- ROM. a floppy disk, a magnetic 
card or the like. 

[0013] Referring to Fig. 1, an image focussed upon a 
CCD 1 by an optical system not shown is converted into 
accumulated charge information which is converted by 
an A/D converter 2 from analog signals into digital sig- 
nals. The digital signal Is transferred via a CPU bus 3 to 
a RAM 4. If frame read of CCD 1 Is performed, the dig- 
ital accumulated charge information is stored in RAM 4 
as shown in Fig. 2. 

[001 4] CPU 5 sets predetermined parameters to a sig- 
nal processing circuit 9, a process SSG circuit 7 and a 
JPEG compression circuit 1 1 . and instructs the process 
SSG circuit 7 to process signals. The predetermined 
parameters include an image data location of a mem- 
ory, an image size, color filter information of CCD 1 . a 
gamma correction value, an Image compression set 
value and the like. 

[0015] The process SSG circuit 7 sets four values, 
including a DMA transfer start address (adr). a horizon- 
tal transfer number (m). a vertical transfer number (n). 



and a vertical offset value (offset) to a two-dimensional 
DMAC (direct memory access controller) 6 to thereby 
read image data from RAM 4. 

[0016] Fig. 3 illustrates the read sequence of image 
5 data from RAM 4 assuming that adr = 0x030 (hereinaf- 
ter, numerals added with Ox are hexadecimal numerals), 
m = 0x8. n * 0x5. and offset = 0x010. 
[0017] The two-dimensional DMAC 6 holds adr 
therein as a read start address, and resets an internal 
JO counter for holding the horizontal read number and 
another internal counter for holding the vertical read 
number, to "0". Each time data is read, the counter for 
holding the horizontal read number is incremented by 
"1". 

IS [0018] When the count of the counter for holding the 
horizontal read number coincides with m. the data read 
start address is incremented by offset, the counter for 
holding the horizontal read number is reset to "0". and 
the counter for holding the vertical read number is incre- 

20 mented by "1 ". Data read is terminated after the counter 
for holding the vertical read number coincides with n. In 
this manner, by sequentially changing the read address, 
data Is read from a rectangular area of RAM 4 as shown 
in Fig. 3. 

25 [001 9] The process SSG circuit 7 repetitively controls 
- the two-dimensional DMAC 6 to divide image data in 
RAM into a plurality of blocks and read bjocks 0. 1. 2, ... 
pg-1 in this order from RAM 4 as shown in Fig. 4. 
[0020] Referring to Fig. 4, H and V represent the hor- 

30 izontal and vertical sizes of image data of one from 
• transferred from CCD 1 to RAM 4. The image data is 
divided into p x q blocks, p blocks in the horizontal direc- 
tion and 9 blocks in the vertical direction, with hatched 
portions being overlapped. 

35 [0021 J Image data read from RAM 4 in the above 
manner is sent via the cpu bus 3 to a delay circuit 8 and 
signal processing circuit 9. A storage capacity of the 
delay circuit 8 is: 

40 2 (lines) x Hb x (the number of bits expressing one 
pixel). 

[0022] Each block shown in Fig. 4 is constituted of four 
areas as shown in Fig. 5. 

45 [0023] Referring to Fig. 5. an area 4-A is an area 
wherein image data read from RAM 4 is valid and image 
data read from the delay circuit 8 is invalid. Therefore, in 
this area, an output from the signal processing circuit 9 
is also invalid. "Vdelay" is equal to the number of 

50 delayed lines. In this embodiment, the Vdelay is "2" 
assuming that the signal processing circuit 9 uses a fil- 
ter with three vertical taps. 

[0024] Areas 4-B and 4-D are areas wherein although 
image data read from RAM 4 and delay circuit 8 is valid. 
55 an output of the signal processing circuit 9 is invalid 
because horizontal image data of two lines exist in the 
filters of the signal processing circuit 9. If it is assumed 
that the signal processing circuit 9 is formed by a circuit 
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constituted of a deiay circuit and a filter having the 
number of NTap of taps in the horizontal direction, the 
length of the areas 4-B and 4-D in the horizontal direc- 
tion is NTap/2. For example, the horizontal length is "5" 
if NTap is "11" because of rounding off the numerals s 
lower than the decimal point. In this embodiment, NTap 
is "11" so that the horizontal length Hfil "5". 
[0025] An area 4-C is an area wherein an outoput of 
the signai processing circuit 9 is valid. The horizontal 
length Ha of this area 4-G is set to a multiple of a hori- io 
zontal direction size of MCU (minimum coded unit 
(MCU): minimum data unit of JPEG), and the vertical 
length Va is set equal to the vertical direction size of 
MCU. Va is therefore "8" for a thinning-out ratio of 4:2:2 
of JPEG. IS 
[0026] As described above, an output of the signal 
processing circuit 9 becomes valid only in a partial area 
of each block. As shown in Fig. 4, the process SSG cir- 
cuit 7 controls the two-dimensional DMAC 16 in such a 
manner that the area 4-A overlaps the area 4-C of the 20 
upper block and the area 4-B overlaps the area 4-0 of 
the left block. 

[0027] The overlap.in the horizontal direction is shown 
in Fig. 6, and that in the vertical direction is shown in 
Fig. 7. 25 
[0028] Referring to Rg. 6, blocks X and X+1 are over- 
lapped in the horizontal direction In this case, Hoffset = 
Ha. 

[0029] Referring to Fig. 7. blocks X and X+p are over- 
lapped in the vertical direction. In this case. Voffset - so 
Va. 

[0030] As shown in Fig. 8, the signal processing circuit 
9 processes image data of one frame and supplies a 
raster/block converter 10 with output blocks 0, 1. ... pq- 
1 in this order. However, image data output from the sig- 35 
nal processing drcuit 9 is smaller than the whole input 
image data by Hfil at right and left end portions in the 
horixontal direction and by Vdelay lines at the upper end 
portion in the vertical direction. 

[0031 ] The size of each output block Is Ha in the hori- 40 
zontal direction and Va in the vertical direction. The 
raster/block converter 10 sequentially and continuously 
converts these output blocks and outputs signals equiv- 
alent to those when image data not divided into blocks 
is supplied. A storage capacity of a line buffer in the 45 
raster/block converter 1 0 is therefore sufficient if such 
the capacity has Ha lines in the horizontal direction, and 
Va lines in the vertical direction. 
[0032] Next, a JPEG compression circuit 1 1 com- 
presses by JPEG method, image date raster/block con- so 
verted by the raster/block converter 10. and writes the 
compressed image data in RAM 4 via the CPU bus 3. 
Upon reception of an instruction from CPU 5, the JPEG 
compressed data written in RAM 4 is recorded in a 
recording medium 13 via the CPU bus 3 and an l/F 12 in " ss 
accordance with a file record format such as FAT. 
[0033] In this embodiment, the buffer capacities 
required for the delay circuit 8 and raster/block con- 



verter 10 are calculated in the following manner, assum- 
ing that input image data has 1034 pixels in the 
horizontal direction and 770 pixels in the vertical direc- 
tion, the horizontal filter of the signal processing circuit 
9 has 1 1 taps, the vertical filter has three taps, the thin- 
ning-out ratio of JPEG is 4:2:2, CCD image data has ten 
bits per pixel, YUV each has eight bits per pixel, and an 
image is divided into 46 blocks in the horizontal direc- 
tion and 96 blocks in the vertical direction: 

Ha = (1034 - Hfil x 2)/4 (division) 
= (1034-5x2)/4 
«256 

Va = 8 

Vb = Va 2 

= 10 bits 
Hb = Ha -H Hfil X 2 

= 256 + 5 X 2 

= 266 

therefore, the buffer capacity of the delay circuit 8 is 

2 X Hb X 10 bit = 5320 bits: and 

the buffer capacity of the raster/block converter is 

Va X Ha X ((number of bits of Y) + (number of bits of 

UV)) = 8 X Ha X ((number of bits of Y) -»- (number of 

bits of U V)) s= 8 X 256 X 1 6 bits 

= 32768 bits. 

The embodiment can therefore reduce the buffer capac- 
ity more than that used by .the conventional process 
described earlier. 

[0034] While image data is read from the area 4-A of 

RAM 4 shown in Fig. 5. the output of the signal process- 
ing circuit 9 is always invalid and the signal output from 
the delay circuit 8 to the signal processing circuit 9 is 
also invalid. Therefore, as shown in Fig. 9, the process 
SSG circuit 7 may control clock signals which drive the 
signal processing circuit 9. and during the period while 
image data in the area 4-A is read from RAM 4, a supply 
of clock signals to the signal processing circuit 9 is 
stopped by a switch 15. Even if such a clock control is 
performed for the signal processing circuit 9, valid out- 
put signals are not affected at all because the clock con- 
trol is performed during the period while invalid signals 
are output. 

[0035] In this embodiment, although parameters and 
two-dimensional DMAC 6 are used, the process SSG 
circuit 7 may control a usual DMAC to read image data 
from RAM 4 in the manner similar to the embodiment. . 
[0036] Fig. 1 0 is a block diagram showing an image 
processing apparatus according to the second embodi- 
ment of the invention. In this embodiment, switches 19 
and 20 and an enlarging and reducing circuit 17 are 
added to the first embodiment apparatus. In Fig, 10, 
same elements as those shown in Fig. 1 are repre- 
sented by using identical reference numerals. Accumu- 
lated charge Information of CCD 1 is A/D converted and 
stored in RAM 4 as shown In Fig. 2. 
[0037] A process SSG circuit 7 sets four values includ- 
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ing a DMA transfer start address adr. a horizontal trans- 
fer number m. a vertical transfer number n. and a 
vertical offset value offset, to a two-dimensional DMAC 
6 to thereby read image data from RAM 4. 
[0038] Fig. 3 illustrates the read sequence of image 
data from RAM 4 assuming that adr = 0x030 (hereinaf- 
ter, numerals added with Ox are hexadecimal numerals), 
m = 0x8. n = 0x5. and offset = 0x010. 
[0039] The two-dimensional DMAC 6 holds adr 
therein as a read start address, and resets an Internal 
counter for holding the horizontal read number and 
another internal counter for holding the vertical read 
number, to "0". Each time data Is read, the counter for 
holding the horizontal read number Is incremented by 
"1" and the read address is also incremented by "1 •". 
[0040] When the count of the counter for holding the 
horizontal read number coincides with m. the data read 
start address is incremented by offset, the counter for 
holding the horizontal read number is reset to "0". the 
counter for holding the vertical read number is Incre- 
mented by T, arxJ the read start address is set as the 
read address. Data read is terminated after the counter 
for holding the vertical read number coincides with n. In 
this manner, by sequentially changing the read address, 
data is read from a rectangular area of RAM 4 as shown 
in Fig. 3. 

[0041 ] The process SSG circuit 7 repetitively controls 
the tv«>-dimenslnal DMAC 6 to divide image data In 
RAM into a plurality of blocks' and read blocks 0,1.2.... 
pg-1 in this order from RAM 4 as shown in Fig. 4. In this 
manner, the image data read from RAM 4 is supplied via 
the CPU bus 3 to a delay circuit 8 and a signal process- 
ing circuit 9. 

[0042] Each of blocks 0 to pq-1 is constituted of four 
areas as shown In Fig. 5. and each block is featured as 
desCTibed earlier. 

[0043] Next, the enlarging and reducing circuit 17 
enlarges or reduces an output of the signal processing 
circuit 9 at an enlargement or reduction factor desig-- 
nated by CPU 5. The structure of this enlarging and 
reducing circuit 17 is disclosed, for example, in Japa- 
nese Patent Application No. 5-227414. A method of cal- 
culating data to be output from the enlarging and 
reducing circuit 1 7 changes according to the sequential 
order of input image data. 

[0044] For example. Fig. 11 shows a phase relation 
between input and output data when image data is 
reduced by 4/9 and when image data is enlarged by 9/4. 
[0045] In the case of reduction by 4/9, the phase of 
input data changes in nine periods and the phase of 
output data changes in four periods as: 

Douti = Dim 

Dout2 = 3/4Din3 1/4Dln4 
DoutS = 2/4Din5 + 2/4Din6 
Dout4 = 1/4Din7 + 3/4Din8 
DoutS = Din 10 



[0046] Similarly, in the case of enlargement by 9/4. the 
phase of input data changes in four periods and the 
phase of output data changes in nine periods as: 

5 Douti s Dim 

Dout2 = 5/9Dim + 4/9Dln2 

Dout3 = 1/9Dim + 8/9Din2 

Dout4 = 6/9Din2 + 3/9Din3 

DoutS = 2/9Din2 + 7/9Din3 
10 Doute = 7/9Din3 + 2y9Din4 

Dout7 s= 3/9Din3 + 6/9Din4 

Dout8 = 8/9Din4 + l/gDinS 

Dou{9 = 4/9Din4 + 5/9Dln5 

Douti 0 = DinS 

15 

[0047] At an M/N enlargement factor. (N+1 ) pieces of 
Input data are required in order to output M pieces of 
data from the enlargement and reducing circuit 17. 
Therefore. Ha and Va defining the size of the area 4-C 
20 are determined in the following manner. 

Enlargement only in Horizontal Direction 

[0048] If the enlarging and reducing circuit 17 

25 enlarges an image by M/N in the horizontal direction to 
resize or square the image, Ha and Va are determined 
so that the image data of Ha x Va processed by and out- 
put from the enlarging and reducing circuit 1 7 has a hor- 
izontal size which is a multiple of the MCU horizontal 

30 direction size and a vertical size equal to the MCU verti- 
cal direction size. Therefore. Ha takes a value provided 
by multiplying a least common multiple of M and the 
MCU horizontal direction size by N/M and adding "1" to 
the multiplication result and Va takes the MCU vertical 

35 direction size. 

[0049] When only the horizontal direction is enlarged, 
both the switches 19 and 20 are turned to the contact 
"1 " side, so that the output of the enlarging and reducing 
drcuit 17 is supplied via the switches 19 and 20 to a 

40 raster/block converter 10. 

[0050] For example, assuming that effective image 
data after signal processing for CCD 1 has 960 (hori- 
zontal) X 768 (vertical) pixels, the image is enlarged by 
16/1 5 in the horizontal direction to form image data hav- 

45 ing 1024 (horizontal) x 768 (vertical) pixels,, and this 
image data Is JPEG compressed through thinning-out 
of 4:2:2. Ha and Va are determined in the following 
manner: 

50 MCU horizontal direction size = 16 pixels, and 
MCU vertical direction size * 8 pixels; 
therefore 

Ha = (least common multiple of 16 and 16)/ 16 x 15 
xn + 1 = l5xn+1(nisa natural number). 
55 and 

Va = 8. 

[0051] The maximum value of n is determined from 
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the buffer capacities of the delay circuit 8 and 
raster/block converter 10. For example, assuming that 
accumulated charge information of one pixel of CCD is 
constituted of 10 bits, the delay circuit 8 has a buffer 
capacity capable of storing two lines of horizontal 640 
pixels the raster/block converter 10 has a buffer capac- 
ity capable of storing eight lines of horizontal 640 pixels, 
and Hfil is "5". then: 

n1 = (640 - 1 - Hfil X 2)/1 5 = 41 (numerals lower than 
decimal point are rounded off); 
n2 - 640(15 x 16/15) = 40 (numerals lower than 
decimal point are rounded off): and 
(maximum value of n) = (smaller one of n1 and n2) 
= 40. 

[0052] Each time the signal processing circuit 9 proc- 
esses one block, the enlarging and reducing circuit 1 7 
outputs to the raster/block converter 10 image data hav- 
ing (16 X n) (horizontal) x 8 (vertical) pixels, i.e., image 
data having a horizontal size which Is a multiple of the 
MCU horizontal direction size and a vertical size which 
is equal to the MCU vertical direction size. 

Reduction only in Horizontal Direction 

[0053] If the enlarging and reducing circuit 1 7 reduces 
an image by M/N in the horizontal direction to resize or 
square the image. Ha and Va are determined so that the 
image data of Ha x Va processed by and output from 
the enlarging and reducing circuit 1 7 has a horizontal 
size which is a multiple of the MCU horizontal direction 
size and a vertical size equal to the MCU vertical direc- 
tion size. Therefore, Ha takes a value provided by multi- 
plying N/M by a least common multiple of M and the 
MCU horizontal direction size, and Va takes the MCU 
vertical direction size. 

[0054] . When only the horizontal direction is reduced, 
both the switches 19 and 20 are turned to the contact 
"1 " side, so that the output of the enlarging and reducing 
circuit 17 is supplied via the switches 19 and 20 to a 
raster/block converter 10. 

[0055] For example, assuming that effective image 
data after signal processing for CCD 1 has 1060 (hori- 
zontal) X 768 (vertical) pixels, the image is reduced by 
28/29 in the horizontal direction to form image data hav- 
ing 1024 (horizontal) x 768 (vertical) pixels, and this 
image data is JPEG compressed through thinning-out 
of 4:2:2. Ha and Va are determined In the following 
manner: 



[0056] The maximum value of n is determined from 
the buffer capacities of the delay circuit 8 and 
raster/block converter 10. For example, assuming that 
accumulated charge information of one pixel of CCD is 
constituted of 10 bits, the delay circuit 8 has a buffer 
capacity capable of storing two lines of horizontal 640 
pixels, the raster/block converter 10 has a buffer capac- 
ity capable of storing eight lines of horizontal 640 pixels, 
and Hfil is "5". then: 
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MCU horizontal direction size * 16 pixels, and 
MCU vertical direction size = 8 pixels; 
therefore 

Ha = (least common multiple of 28 and 16)/28 x 29 
xn = n6xn(nisa natural number), 
and 
Va = 8. 



n1 = (640 - 2 x Hfil)/1 16=5 (numerals lower than 
decimal point are rounded off); 
n2 = 640/(116 x 28/29) = 5 (numerals lower than 
decimal point are rounded off); and 
IS (maximum value of n) = (smaller one of n1 and n2) 
= 5. 

[0057] Each time the signal processing circuit 9 proc- 
esses one block, the enlarging and reducing circuit 17 
20 outputs to the raster/block converter 1 0 image data hav- 
ing (7 X 16 X n) (horizontal) x 8 (vertical) pixels, i.e.. 
image data having a horizontal size which is a multiple 
of the MCU horizontal direction size and a vertical size 
equal to the MCU vertical direction size. 

Enlargement in Vertical Direction by Devisor of MCU 
Vertical Direction Size . 

[0058] If the enlarging and reducing circuit 17 
enlarges an image by a divisor of the MCU vertical 
direction size in the vertical direction, Ha Is determined 
in the manner similar to that In 

"Enlargement only in Horizontal Direction" or . 
"Reduction only in Vertical Direction". 

[0059] The block vertical size Va Is determined as in 
the following: 

If the MCU vertical direction size is "8", then 
Va = 8 at an enlargement factor "1 " 
Va = 4 at an enlargement factor "2" 
Va = 2 at an enlargement factor "4" 
Va = 1 at an enlargement factor "8"; whereas if the 
45 MCU vertical direction size is "16", then . 
Va = 16 at an enlargement factor "1 " 
Va.= 8 at an enlargement factor "2" 
Va = 4 at an enlargement factor "4" 
Va = 2 at an enlargement factor "8" 
so Va = 1 at an enlargement factor "16". 

[0060] In this case, both the switches 19 and 20 are 
turned to the contact "1 " side, so that the output of the 
enlarging and reducing circuit 17 is supplied via the 
55 switches 19 and 20 to raster/block converter 10. Each 
time the signal processing circuit 9 processes one 
block, the enlarging and reducing circuit 1 7 outputs to 
the raster/block converter 1 0 image data having a hori- 
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zontal size which is a multiple of the MCU horizontal 
direction size and a vertical size equal to the MCU verti- 
cal cfirection size. 

Reduction in Vertical Direction by 1/n - s 

[0061 ] It the enlarging and reducing circuit 1 7 reduces 
an Image by 1/n. Ha is determined in the manner similar 
to that in "Enlargement only in Horizontal Direction" or 
"Reduction only in Vertical Direction". 

Va is determined as: 

Va = (MCU vertical direction size) x n. 

[0062] In this case, both the switches 19 and 20 are is 
turned to the contact "1" side, so that the output of the 
enlarging and reducing circuit 17 is supplied via the 
switches 19 and 20 to a raster/block converter 10. Each 
time the signal processing circuit 9 processes one 
block, the enlarging and reducing circuit 17 outputs to so 
the raster/block converter 10 image data having a hori- 
zontal size which is a multiple of the MCU horizontal 
direction size and a vertical size equal to the MCU verti- 
cal direction size. 

25 

Enlargement/Reduction in Vertical Direction other than 
Enlargement by Divisor of MCU Vertical Direction Size 
and Reduction by 1/n 

[0063] If the enlarging and reducing circuit 17 30 
enlarges in the vertical direction, an image by Mv/Nv 
other than a divisor of the MCU vertical direction size 
and 1/n and enlarges or reduces the image by Mh/Nh in 
the horizontal direction. Ha is deternnined as: 

35 

Ha = (multiple of Nh) -i- 1 for enlargement: and 
Ha = (multiple of Nh) for reduction. 

[0064] Va is determined as a multiple of Va. In this 
case. CPU 5 operates to make both the switches 1 9 and 40 
20 be turned to the contact "0" side, so that the output of 
the signal processing circuit 9 is enlarged or reduced by 
the enlarging and reducing circuit 17 and thereafter 
temporarily stored in RAM 4. The image data written in 
RAM 4 is divided in the unit of MCU vertical direction 45 
size and supplied via the switch 20 the raster/block con- 
verter 1 0. 

[0065] By determining Ha and Va and controlling the 
switches 19 and 20 in the manner described above, an 
image data having a horizontal size which is an integer so 
multiple of the MCU horizontal direction size and a ver- 
tical size which is equal to the MCU vertical direction 
size, is output to the raster/block converter 10. 
[0066] The process SSG circuit 7 controls the two- 
dimensional DMAC 6 in such a manner that areas of ss 
each block overlap as shown in Fig. 6 in the horizontal 
direction and in Fig. 7 in the vertical direction. 
[0067] In Fig. 6. Hoffset is: 



Hoffset - Ha (for reduction); and 
Hoffset = Ha - 1 (for enlargement). 

For the reduction, the areas 4-C of the blocks X and X+i 
become continuous in the horizontal direction, and for 
the enlargement, the areas 4-C of the blocks X and X+i 
overlaps by one pixel In the horizontal direction. 
[0068] In Rg. 7. Voffest is: 

Voffset = Va. 

The areas 4-C of the blocks X and X+p become contin- 
uous In the vertical direction. 

[0069] With the atjove operations, even if the image is 
divided and processed in the unit of block by the enlarg- 
ing and reducing circuit 1 7. distortion of the image to be 
caused by phase differences is not formed. 
[0070] In the second embodiment. Ha is determined 
from an enlargement or reduction factor. This factor may 
be deterniined from the buffer capacities of the delay 
circuit 8 and raster/block converter 10. 
[0071 ] For example, assuming that the buffer capacity 
of the raster/block converter 10 is 320 pixels in the hori- 
zontal direction, a target enlargement factor is 11/10. 
and the MCU horizontal direction size is 16. the horizon- 
tal direction size Ha of the area 4-C of a block which 
makes the buffer memory of the raster/block converter 
10 full, is: 

Ha = 320 X 10/1 1 = 291 (numerals lower than deci- 
mal point are rounded off). 

[0072] In this case, the enlargement factor of the 
enlarging and reducing circuit 17 is set to 320/291 so 
that the number of horizontal pixels in an output of the 
enlarging and reducing circuit 1 7 is 320 which satisfies 
the condition of a multiple of the MCU horizontal direc- 
tion size. Although this factor is different from the target 
enlargement factor, this precision does not pose any 
practical problem. 

[0073] Fig. 12 is a block diagram showing .an image 
processing apparatus according to the third embodi- 
ment of the present invention. In this embodiment, a 
phase buffer 21 is provided which is connected to a 
process SSG circuit 7 and an enlarging and reducing 
circuit 1 7. In Fig. 1 2. the same elements as those shown 
in Fig. 10 are shown by using identical reference numer- 
als. 

[0074] Image information of CCD 1 stored in RAM 4 
as shown in Fig. 2 is read by the process SSG circuit 7 
and two-dimensional DMAC 6 in the order illustrated in 
Fig. 3. In this case, in the two-dimensional DMAC 6. the 
operations of an internal counter for holding the horizon- 
tal read number and another internal counter for holding 
the vertical read number, are performed in the manner 
similar to the second emtxxdiment illustrated in Fig. 10. 
By changing the read address of RAM 4. image data is 
read from a rectangular area of HAM 4 as shown in Fig. 
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3. 

[0075] As shown in Fig. 4, image data divided into a 
plurality of blocks is read from RAM 4 in the order of 
blocks 0. 1.2, .... pg-1 . The image data read from RAM 
4 is supplied via a CPU bus 3 to a delay circuit 8 and a 
signal processing circuit 9. Each of blocks 0 to pq-l is 
constituted of four areas as shown in Fig. 5. 
[0076] Similar to the embodiment shown in Fig. 10, 
the enlarging and reducing circuit 17 calculates output 
data by a different method depending upon the sequen- 
* tial order of input image data. 

[0077] For example, the phase relations between 
input and output data at a reduction factor of 1/9 and at 
an enlargement factor of 9/4 are set in the manner as 
illustrated in Fig. 11.. 

[0078] As described earlier, in order for the enlarging 
and reducing circuit 17 to output M pieces of data at an 
enlargement factor of M/N. N+1 pieces of input data are 
necessary. Therefore, depending upon the width of Ha. 
the phase in the enlarging and reducing circuit 17 may 
become discontinuous when the block changes, and 
image distortion is generated. 

[0079] In order to avoid this, in this embodiment, the 
phase in the enlarging and reducing circuit 17 at the 
boundary of adjacent blocks and image data are stored 
in the phase buffer. When the block changes, the phase 
at the boundary of geometrically adjacent blocks and 
innage data are read to perform an interpolation opera- 
tion for enlargement and reduction. 
[0080] The operation of storing and reading the phase 
and image data at the block boundary will be described. 
[0081] Fig. 13 shows the structures of the enlarging 
and reducing circuit 1 7 and phase buffer 21 . The enlarg- 
ing and reducing circuit 1 7 is constituted by a horizontal 
enlarging and reducing circuit 171 and a vertical enlarg- 
ing and reducing circuit 172, and the phase buffer 21 is 
constituted by a horizontal phase buffer 21 1 and a verti- 
cal phase buffer 212. 

[0082] Fig. 14 shows the structures of an enlarging 
and reducing circuit 66 constituting each enlarging and 
reducing circuit 171, 172 and a phase buffer 67 consti- 
tuting each phase buffer 211, 212. In Fig. 14. SIG-IN. 
STO, LOAD, and SEL are control signals supplied from 
the process SSG circuit 7. The enlarging and reducing 
circuit 66 reads image data via SIG-iN and an interpola- 
tion operation is performed by an interpolation operation 
circuit 61 . A phase for such the interpolation operation is 
obtained from the phase counter 62. 
[0083] When STO becomes active (in the following 
description, the control signal takes an active high 
level), image data, image data at two points for linear 
interpolation, stored in the interpolation operation circuit 
61 is output via a DAT terminal of the interpolation oper- 
ation circuit and WEN (write enable) is made active. At 
this time, image data is written in a buffer memory 65 
selected by a switch 64 in accordance with the value of 
SEL Also at this time, a count indicating a current 
phase and output via a CNT terminal of the phase coun- 



ter 62 is selected by the switch and written in the buffer 
memory 65. 

[0084] When LOAD becomes active, the phase and 
image data stored in the buffer memory 65 selected by 
5 . a switch 63 in accordance with the value of SEL are out- 
put to the phase counter 62 and interpolation operation 
circuit 61 . When LOAD becomes active, the interpola- 
tion operation circuit 61 supplied the image data to the 
interpolation operation circuit 61 via a LOAD-DAT termi- 
10 nal. When LOAD becomes active, the phase counter 62 
reads the phase via a LOAD-CNT terminal. 
[0085] The control to be executed by the process SSG 
circuit 7 relative to the horizontal enlarging and reducing 
circuit 171. horizontal phase buffer 21 1. vertical enlarg- 
es ing and reducing circuit 1 72. and vertical phase buffer 
. 212 operating in the above manner will be described 
with reference to Fig. 15. 

[0086] In Fig. 15. HSEL, HSTO. and HLOAD are SEL. 
STO, and LOAD of the horizontal enlarging and reduc- 
20 ing circuit 171 and horizontal phase buffer 211. VSEL, 
VSTO. and VLOAD are SEL, STO. and LOAD of the ver- 
tical enlarging and reducing circuit 172 and vertical 
phase buffer 212. 

[0087] HLOAD is made active before the horizontal 

25 enlarging and reducing circuit 171 processes image 
data at the top of a line of each block, and HSEL is set 
to the value same as the line number in the block of cur- 
rent image data. Therefore, the horizontal enlarging and 
reducing circuit 171 reads from the horizontal phase 

30 buffer 211 the image data and phase at the end of the 
same line in the preceding block. 
[0088] HSTO is made active after the horizontal 
enlarging and reducing circuit 171 processes image 
'data at the end of a line of each block, and HSEL is set 

35 to the value same as the line number in the block of cur- 
rent image data. Therefore, the image data and phase 
at the end of the current line is read out from the hori- 
zontal enlarging and reducing circuit 171 and then writ- 
ten into the horizontal phase buffer 211. 

40 [0089] VLOAD is made active while the enlarging and 
reducing circuit 172 processes image data of the first 
line of each block, and VSEL is set to the v^lue indicat- 
ing the horizontal pixel position of the image data to be 
processed in the block. Therefore, the vertical enlarging 

45 and reducing circuit 172 reads from the vertical phase 
buffer 212 the image data at the same horizontal pixel 
position one fine before. 

[0090] VSTO is made active while the enlarging and 
reducing circuit 172 processes image data of the last 

50 fine of each block, and VSEL is set to the value indicat- 
ing the horizontal pixel position of the image data to be 
processed in the block. Therefore, the image data and 
phase at the current line is read out from the vertical 
enlarging and reducing circuit 172 and then written into 

£5 the vertical phase buffer 212. 

[0091] The horizontal phase buffer 21 1 is reset when 
the enlargement/reduction is completed for the block p- 
1, 2p-l pq-1 shown in Fig. 4. i.e., tfie last horizontal 
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block. The vertical phase buffer 212 is reset when the 
enlargement/reduction is completed for the block pq-1 
shown in Fig. 4, i.e.. the last block of one frame. 
[0092] As the process SSG circuit 7, enlarging and 
reducing circuit 17 and phase buffer 21. operate in the 
manner described above, enlargement/reduction with- 
out distortion can be performed through pipelining 
processings by hardware even if image data is divided. 
[0093] An output of the enlarging and reducing circuit 
17 is input to a raster/block converter 10. The 
raster/block converter 10 performs a raster/block con- 
version of the image data having a horizontal size which 
Is a multiple of the MCU horizontal direction size and a 
vertical size equal to the MCU vertical direction size. Ha 
and Va of the area 4-C are therefore determined so that 
the image data enlarged/reduced by the enlarging and 
reducing circuit 1 7 has a horizontal size which is .a mul- 
tiple of the MCU horizontal direction size and a vertical- 
size equal to the MCU vertical direction size. 
[0094] The buffer capacity of the phase buffer 2 1 nec- 
essary for storing the phase and Image data is deter- 
mined as in the following. For example, assuming that 
the image enlarged/reduced has 1024 horizontal pixels, 
one pixel is constituted of 16 bits, the phase is 
expressed by 8 bits, the fVlCU vertical direction size is 8. 
and the reduction is performed at a reduction factor of 
4/5 in the vertical direction, the buffer capacities are: 

{ 1 6 8) X 8 X 5/4 = 240 bits for the horizontal phase 
buffer 211; and 

(16 8) X 1024 = 24576 bits for the vertical phase 
buffer 212. 

[0095] Fig. 16 is a block diagram showing an image 
processing apparatus according to the fourth embodi- 
ment of the invention. A different point from the third 
embodiment is a provision of switches 19 and 20. 
[0096] In this embodiment, as shown in Fig. 1 7, an 
enlarging and reducing circuit 17 and a phase buffer 21 
are constituted by a horizontal enlarging and reducing 
circuit 91 . a vertical enlarging and reducing circuit 93. 
and a horizontal phase buffer 92. 
[0097] If CCD 1 has non-square pixels, the enlarging 
and reducing circuit 17 squares each pixel by enlarging 
or reducing only in the horizontal direction. In tfiis case, 
the block division of image data is performed in the 
mariner similar to the third embodiment, and CPU 5 
turns the switches 19 and 20 to the contact "1" side. 
[0098) The enlarging and reducing circuit 17 can 
therefore output an image having a horizontal size 
which is a multiple of the MCU horizontal direction size 
and a vertical size equal to the MCU vertical direction 
size. An output of the enlarging and reducing circuit 17 
is input via the switches 19 and 20 to a raster/block con- 
verter 10 to be raster/block converted, and thereafter 
JPEG compressed by a JPEG compression circuit ii. 
The JPEG compressed data is written in RAM 4. 
[0099] If the image data is not only squared bU also 



resized, CPU 5 turns the switches 1 9 and 20 to the con- 
tact "0" side. Of Ha and Va of the area 4-C of each 
block. Ha is set to an arbitrary value which does not 
exceed the allowable range determined by the buffer 
5 capacity of the delay circuit 8. and Va is set as Va = Nv 
X n (n is a natural number) at an enlargement factor of 
Mv/Ny at the enlarging and reducing circuit 17. 
[01 00] As described with *he third embodiment, of the 
enlarging and reducing circuit 17, the horizontal enlarg- 
• 10 ing and reducing circuit 91 realizes a continuity of the 
phase thereof by reading the preceding image data and 
phase from the phase buffer 21. whereas the vertical 
enlarging and reducing circuit 93 realizes a continuity of 
the phase thereof by setting Va to a common multiple of 
75 Nv. 

[01 01 ] An output of the enlarging and reducing circuit 
17 is written via the switch 19 and CPU bus 3 into RAM 
4. The image data enlarged/reduced and stored in RAM 
4 is supplied via the CPU bus 3 and switch 20 to the 
20 raster/block converter 10 to be raster/block converted, 
and thereafter JPEG compressed by the JPEG com- 
pression circuit 11. The JPEG compressed data is writ- 
. ten in RAM 4. 

[01 02] As described above, according to the embodi- 
es ments, image data is processed after it is divided into 
blocks. Therefore, the buffer capacity necessary for the 
succeeding processes such as processes by a delay 
circuit and a raster/block converter can be reduced. 
[0103] Further, the horizontal pixel size allowing 
30 image signal processing does not depend upon the 
capacities of buffer memories to be used by the delay 
circuit and raster/block converter. Therefore, it is possi- 
ble to process image data having an arbitrary horizontal 
pixel size. It is therefore possible to reduce the memory 
35 capacity and cost. 

[0104] Image data having an arbitrary size can be 
processed and compressed without using a memory 
such as RAM so that high speed signal processing is 
possible. 

40 [0105] According to the embodiments, image data 
read from RAM or the like and compressed through 

JPEG or the like can be processed through hardware 
pipelining, so that high speed signal processing is pos- 
sible. 

45 [01 06] Since the number of accesses to RAM can be 
reduced, a power required for signal processing can be 
saved. 

[0107] Image data having an arbitrary size can be 
enlarged or reduced independently from the capacities 
50 Of the buffer memories. It is therefore possible to reduce 
the memory capacity and cost. 

[0108] Image squaring including signal processing 
and compressing can be performed through hardware 
pipelining without using a memory such as RAM. 
55 [0109] Still further, clocks used for signal processing 
can be stopped during a specific period so that a power 
required for signal processing can be saved. 
[01 1 0] In the above embodiments, a JPEG scheme is 
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used as a compression coding method. The invention is 
not limited only thereto, but is applicable to all coding 
systems for executing coding on a predetermined block 
unit basis (e.g.. a MPEG scheme). 
[0111] In other words, the foregoing description of 
embodiments has been given for illustrative purposes 
only and not to be construed as imposing any limitation 
in every respect. 

[0112] The scope of the invention is, therefore, to be 
determined solely by the following claims and not lim- 
ited by the text of the specifications and alterations 
made with a scope equivalent to the scope of the claims 
fall within the true spirit and scope of the Invention. 

Claims 

1. An image processing apparatus comprising: 

input means for inputting image data; 
dividing means for dividing the input irnage 
data into blocks, said dividing means dividing 
the input image data so that adjacent blocks 
partially overlap the input image data; and 
processing means for filtering the image data 
divided by said dividing means in a block unit 
basis. 

2. An image processing apparatus according to claim 

1, further comprising coding means for compres- 
sion coding an output of said processing means. 

3. An image processing apparatus according to claim 

2, wherein a horizontal size of the block is deter- 
mined from a horizontal size of one frame of the 
image data and the number of taps of a horizontal 
filter to be used by said processing means, and a 
horizontal direction length of an overlapped portion 
of the block is determined from the number of taps 
of the horizontal filter. 

4. An image processing apparatus according to claim 

3, wherein the horizontal size of the block is deter- 
mined from a horizontal size of a minimum process- 
ing unit of said coding means. 

5. An image processing apparatus according to claim 
3, wherein a vertical size of the block is determined 
from the number of taps of a vertical filter to be used 
by said processing means and an operation mode 
of said coding means, and a vertical direction 
length of an overlapped portion of the block is 
determined from the number of taps of the vertical 
filter. 

6. An image processing apparatus according to claim 
1. further comprising: 

converting means for enlarging or reducing an 



output of said processing means; and 
coding means for compression coding an out- 
put of said converting means. 

5 7, An image processing apparatus according to claim 

6, wherein a horizontal size of the block is deter- 
mined from a horizontal size of one frame of the 
image data, the number of taps of a horizontal filter 
to be used by said processing means and an 

7(7 enlargement factor, in a horizontal direction of said 
converting means, and a horizontal direction length 
of an overlapped portion of the block Is determined 
from the number of taps of the horizontal filter and 
the enlargement factor in the horizontal direction of 

IS said converting means. 

8. An image processing apparatus according to claim 

7, wherein a vertical size of the block is determined 
from the number of taps of a vertical filter to be used 

20 by said processing means and an operation mode 
of said coding means, and a vertical direction 
length of an overlapped portion of the block is 
determined from the number of taps of the vertical 
filter. 

25 

9. An image processing apparatus according to claim 

1 . further comprising: 

delay means for delaying the image data to be 
30 supplied to said processing means; and 

control means for stopping an operation clock 
to be supplied to said processing means, dur- 
ing a period while an overlapped portion in a 
vertical direction is written in said delay means, 
35 sequentially from a first block. 

10. An image processing apparatus according to claim 

2, wherein said coding means performs a coding 
process in conformity with a JPEG scheme. 

40 

11. An image processing apparatus according to claim 
6, wherein said coding means performs a coding 
process In conformity with a JPEG scheme. 

45 12. An image processing apparatus according to claim 
1, wherein said dividing means includes memory 
means for storing the image data, and the image 
data Is divided into blocks by reading the image 
data from the memory means through two-dimen- 

50 sional DMA. 

13. An image processing apparatus according to claim 
1, wherein said input means including image pickup 
means for picking up an object image and output- 

£5 ting the image data. 

1 4, An image processing apparatus according to claim 
8. wherein said dividing means includes memory 
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means for storing the image data, and the image 
data is read from the memory means in the block 
unit basis. 

1 5. An image processing apparatus according to claim s 
14. wherein an output of said converting means is 
temporarily stored in the memory means and then 
supplied to said coding means. In a case other than 
enlargement at an enlargement factor equal to a 
divisor of a vertical size of a minimum processing io 
unit In a vertical direction of said coding means or 
reduction at a reduction factor of one divided by a 
positive integer. 

16. An image processing method comprising the steps 75 
of: 

inputting image data; 

dividing the input image data into blocks, said 
•dividing step dividing the Input image data so 20 
that adjacent blocks partially overlap the image 
data; and 

filtering the divided image data In a block unit 
basis. 

25 

17. A computer readable storage medium storing an 
image processing program, the program compris- 
ing: 

an input process of inputting image data; so 
a dividing process of dividing the input irhage 
data into blocks, said dividing process dividing 
the input image data so that adjacent blocks 
partially overlap the image data; and 
a process of filtering the divided image data In 35 
a block unit basis. 
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(57) Input image data is divided into blocks so as to 
make adjacent blocks partially overlap image data, and 
the divided image data Is filtered in the unit of block. An 
image processing apparatus and method are provided 
which can process image data of an arbitrary size at 



high speed independently from the capacity of a buffer 
memory, and a computer readable storage medium is 
provided which stores a program realizing such a 
.method. 
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